Evaluating Outcomes

Assessment and Education Research 2 7 The April 2004 CCLI conference focused on building excellence in undergraduate STEM education. The question motivating this chapter (and the workshop on which it is based) was posed by a member of the NSF staff during electronic discussions before the CCLI conference. “... You decided to develop new curriculum materials, or to adapt and implement them in your institution ... What evidence enables (or will enable) you to infer that it is working—and why?” The educational context in which the motivating question is supposed to be asked and answered is often seen to be fraught with insurmountable problems. • What measures should be taken? (Grades? Ratings? Quality of problem solutions?) • What comparisons should be made? • What is a suitable control group? • What if a control group is not possible? (There is only one class!) • Students are so different. How can you make sense of the data? In addition to the above generic questions (that I often hear), another question and a comment surfaced in the audience reaction to the CCLI workshop. • How does one motivate colleagues that assessment is important and can be done to provide useful information? • Most scientists have no clue as to what is an acceptable design for educational research. (Scientists are not alone in this puzzlement.) In this chapter, I hope to persuade the reader of three things. First, all these questions can be sensibly answered. Second, with some advance planning, an adequate research design can be devised. And, third, an acceptable evaluation can be conducted.